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ABSTRACT 


On  24-27  June  1992,  the  Association  for  Computing  Machinery  (ACM)  Special 
Interest  Group  for  Ada  (SIGAda)  Artificial  Intelligence  Working  Group  (A1WG)  held  a 
workshop  to  discuss  Ada  real-time  and  Artificial  Intelligence  (AI)  issues.  Workshop 
discussions  covered  blackboard  architectures,  experiences  and  lessons  learned,  real-time 
development  approaches  and  issues,  and  Ada  9X  issues  for  AI  systems.  These  proceedings 
include  papers  by  workshop  participants  describing  their  large-scale  AI  with  Ada  systems.  A 
summary  of  the  related  workshop  experiences  and  lessons  learned  discussions  in  the  areas  of 
requirements  analysis,  design  methodologies,  development  techniques,  test  and  validation, 
and  maintainability  are  included. 
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IV 


EXECUTIVE  SUMMARY 


Artificial  Intelligence  (AI)  with  Ada  is  a  reality!  Based  on  the  information  presented 
at  the  1992  Summer  Association  for  Computing  Machinery  (ACM)  Special  Interest  Group 
for  Ada  (SIGAda)  Artificial  Intelligence  Working  Group  (AIWG)  workshop,  Ada  is  a  viable 
language  for  AI  applications.  This  workshop  provided  a  valuable  opportunity  to  accumulate 
more  empirical  evidence  proving  that  Ada  is  being  used  successfully  to  implement  large- 
scale  AI  systems.  Congratulations  to  the  participants  for  their  AI  with  Ada  successes! 


Workshop  Highlights 

The  SIGAda  AIWG  held  a  very  busy  and  informative  workshop  in  conjunction  with 
the  Summer  '92  SIGAda  meeting  in  Seattle,  Washington.  The  workshop  focus  was  Ada  real¬ 
time  and  Artificial  Intelligence  (AI)  issues.  Software  researchers  and  practitioners  from  the 
real-time  and  AI  communities  were  brought  together  to  exchange  ideas  and  lessons  learned. 
Most  of  our  workshop  time  was  devoted  to  presentations  by  those  who  have  implemented 
large  real-time  AI  with  Ada  systems.  A  summary  of  these  AI  with  Ada  applications  is  shown 
in  figure  1  with  the  size  in  Thousands  of  Source  Lines  of  Code  (KSLOCl  and  the  number  of 
Knowledge  Based  Systems  (KBS)  rules. 
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Figure  1:  Summary  of  AI  with  Ada  Applications 


The  acronyms  used  in  the  figure  are  Data  Fusion  Technology  Demonstrator  System 
(DFTDS),  Ada  Real-Time  Inference  Engine  (ARTIE),  tactical  cockpit  mission  manager 
(ARTIE  MM),  search  area  planner  (ARTIE  SP),  automated  sensor  manager  (ARTIE  SM), 
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and  Training  Control  and  Evaluation  (TC&E).  These  systems  are  a  sampling  of  larger 
applications  than  most  of  those  documented  in  the  1991  A1VVG  applications  survey1 . 


Workshop  participants  shared  their  experiences  implementing  large-scale  real-time 
Knowledge- Based  Systems  (KBSs)  in  Ada.  Participants  from  the  Defense  Research  Agency 
(DRA)  in  the  United  Kingdom  (UK)  shared  their  experiences  developing  a  real-time  ship 
borne  Command  and  Control  system  and  their  research  activities  in  the  area  of  validation  and 
verification  of  safety-critical  KBSs.  A  pair  of  related  articles  describing  their  activities.  The 
Data  Fusion  Technology  Demonstrator  System  (DFTDS)  Project  by  John  Miles  and 
Validation  and  Verification  of  Safety  Critical  Knowledge-Based  Systems  by  Jonathan  Haugh 
are  included  in  these  proceedings.  DFTDS  is  undergoing  sea  trials  at  the  present  time. 

Boeing  provided  an  update  on  the  latest  developments  with  their  Ada  Real-Time 
Inference  Engine  (ARTIE)  and  described  some  of  their  complex  real-time  reasoning  based 
avionics  applications  including  a  tactical  cockpit  mission  manager,  a  search  area  planner,  and 
an  automated  sensor  manager.  George  (Rick)  Wilber  and  Robert  Ensey  provided  a  copy  of 
their  briefing  slides  which  are  enclosed  in  these  proceedings  as  the  article  Embedded  Real- 
lims-Rfiasgning  Congepis  araLAppIisaiippa* 

BBN  has  developed  an  AI  with  Ada  application  called  Training  Control  &  Evaluation 
(TC&E).  Their  experience  provides  some  insight  into  rapid  prototyping  with  Ada  followed 
by  the  transition  of  a  legacy  prototype  to  Full  Scale  Engineering  Development  (FSED). 
Although  Dan  Massey  was  unable  to  attend  the  workshop,  he  provided  an  article  which  is 
titled  Training  Control  &  Evaluation  (TC&E1  for  these  proceedings. 

Minutes  of  the  workshop  have  been  written  in  the  form  of  a  workshop  discussion 
summary  which  is  included  with  the  Message  from  the  Panel  Chair  for  the  Applications 
Experiences  and  Lessons  Learned  Panel.  A  workshop  participant  list  follows  this  Executive 
Summary.  Recommendations  for  specific  AIWG  actions  are  made  at  the  end  of  this 
Executive  Summary. 


Issues 


The  panel  discussed  issues  covering  the  complete  software  life  cycle  including  the 
challenges  of  software  engineering  AI  applications;  the  difficulties  with  AI  requirements 
analysis;  the  legacy  of  an  AI  requirements  analysis;  testing,  verification,  and  validation  of  AI 
applications;  and  maintenance  for  AI  applications.  Many  of  the  issues  and  problems 
discussed  during  the  workshop  are  applicable  to  AI  applications  developed  with  any 
language  including  the  Ada  programming  language. 


1  J.  Johns,  June  1992,  1991  Annual  Report  for  the  ACM  Special  Interest  Group  for  Ada 
Artificial  Intelligence  Working  Group,  MITRE  Document  M92B0000056  and  the 
January/February  issue  of  Ada  Letters. 
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Software  Engineering 


AI  applications  typically  begin  with  few  documented  requirements  and  are  defined  as 
well  as  developed  with  a  series  of  evolutionary  prototypes.  This  is  the  current  sta^e-of-the-art 
for  AI  applications  design  and  development  with  Ada  or  any  other  programming  language. 
Software  engineering  and  Ada  design  methodologies  typically  begin  with  a  set  of  well- 
defined,  testable,  verifiable  requirements.  Therefore,  due  to  the  lack  of  requirements, 
software  engineering  is  a  challenge  for  AI  with  Ada  developers.  Without  a  set  of  well- 
defined  requirements,  how  do  you  "engineer"  AI  applications?  This  question  is  the  subject  of 
on-going  debates  and  research.  However,  while  there  are  no  textbook  solutions  today,  the 
information  in  these  proceedings  describes  how  AI  with  Ada  practitioners  are  successfully 
"engineering"  AI  applications  today. 


AI  Requirements  Analysis 

Requirements  for  AI  applications  are  defined  through  iteration;  that  is,  learning  by 
doing.  AI  requirements  are  difficult,  if  not  impossible,  to  specify  without  prototypes.  Most 
of  the  effort  for  AI  projects  is  spent  defining  requirements  and  prototyping.  In  the  case  of  a 
70,000  source  line  of  code  Ada  application,  2/3  of  the  calendar  time  and  labor  were  expended 
developing  prototypes  to  define  the  requirements  for  the  AI  application. 

AI  requirements  analysis  requires  a  flexible  process  because  defining  AI  requirements 
is  an  evolutionary  process.  This  type  of  process  is  not  a  fixed  price  problem,  and  the  typical 
use  of  DOD-STD-2167A  may  be  too  rigid  for  A!  applications  even  w'ith  the  software 
engineering  discipline  provided  by  the  Ada  programming  language.  We  discussed  many  A I 
specific  issues  associated  with  the  software  engineering  process  and  DOD-STD-2167A. 
These  discussions  led  us  to  question  whether  AI  requirements  and  design  are  the  same  as 
understood  in  the  DOD-STD-2167A  and  other  software  engineering  environments.  Based  on 
these  discussions,  the  AIWG  decided  to  be  more  active  in  expressing  AI  specific 
requirements  and  concerns  to  the  standards  bodies  that  are  developing  standards  which 
impact  the  development  of  AI  applications  with  the  Ada  programming  language. 

The  Legacy  of  AI  Requirements  Analysis 

Prototypes  and  human  understanding  of  the  problem  domain  are  the  standard  legacy 
of  an  AI  requirements  analysis  effort.  What  happens  to  this  legacy?  We  would  like  to  use 
our  human  understanding  to  specify  the  desired  system  in  clear  unambiguous  requirements, 
but  this  is  rarely  possible.  Prototypes  typically  reflect  our  understanding  of  how  to 
implement  a  solution,  and  in  many  cases  are  required  to  prove  that  it  is  possible  to  "engineer" 
the  full  scale  system.  Can  you  successfully  "re-engineer"  a  prototype  into  a  supportable  and 
maintainable  system?  Based  on  panel  experiences  described  in  these  proceedings.  Ada 
prototypes  are  being  re-engineered  into  supportable  and  maintainable  systems. 
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Testing,  Verification,  and  Validation 


The  difficulties  inherent  in  AI  requirements  analysis  inevitably  lead  to  problems  with 
the  testing,  verification,  and  validation  (V&V)  of  Al  applications.  Testing  and  validation 
activities  ensure  that  the  developed  software  system  satisfies  a  well-defined  set  of 
requirements  which  unfortunately  do  not  normally  exist  for  AI  applications.  Verification 
activities  ensure  that  the  developed  system  is  supportable  and  maintainable  which  raises 
issues  associated  with  the  feasibility  of  verifying  non-deterministic  systems  that  can  leam  and 
adapt  over  time.  Testing  and  V&V  are  critical  areas  of  current  research  because  the  public 
and  the  software  engineering  community  have  begun  to  focus  attention  on  building  trusted 
systems  that  are  correct,  dependable,  robust,  safety  critical,  efficient,  and  secure 

The  panel  discussions  regarding  testing  and  V&V  led  to  two  interesting  thoughts. 
First,  AI  applications  seek  to  emulate  human  intelligence  and  behavior.  How  do  we  test  and 
validate  humans?  In  general,  the  proof  of  human  intelligence  and  their  ability  to  learn  is 
through  on-the-job  performance.  Therefore,  in  essence,  humans  do  not  undergo  the  same 
scrutiny  of  test  and  V&V  that  we  are  trying  to  impose  on  AI  applications.  Second,  instead  of 
trying  to  perform  an  unnatural  test  and  V&V  scrutiny  of  Al  applications,  perhaps  it  would  be 
more  meaningful  to  develop  techniques  and  tools  that  determine  if  an  AI  application  is  fit- 
for-purpose;  that  is,  does  it  match  the  problem  domain,  is  it  operable  in  the  proposed 
environments,  and  can  it  be  learned  with  relative  ease. 


Maintenance 

Maintenance  is  an  area  of  critical  concern  as  large-scale  AI  applications  are  fielded 
and  must  be  supported  for  a  long  life  cycle.  Knowledged-base  maintenance  is  also  a  critical 
concern  during  the  iterative  development  of  large  knowledge-based  systems.  For  an  expert 
system,  maintenance  traditionally  involves  changes  to  the  rule  base  as  well  as  the  facts  which 
art  normally  considered  data.  The  discussions  in  this  area  included  questions  such  as  "What 
is  knowledge?  Are  rules  considered  data  or  software?  Should  facts  in  a  knowledge-based 
system  be  treated  as  data  or  software? 

During  the  panel  discussions,  several  implementation  techniques  currently  being  used 
for  expert  systems  were  described  as: 

1.  Implementing  the  rules  in  Ada  for  runtime  performance. 

2.  Use  two  modes  for  the  rule  base:  an  interpretive  mode  for  rale  execution 
during  development  and  a  runtime  mode  that  involves  translating  the  rules  to 
Ada  for  runtime  performance. 

3.  A  runtime  mixture  of  interpreted  rules  and  rules  implemented  in  Ada. 

One  of  the  developers  who  is  using  the  first  approach  of  implementing  the  rules  in 
Ada  lamented  about  the  tremendous  overhead  -  approximately  1  week  --  to  rebuild  his  real¬ 
time  command  and  control  system  for  any  changes  to  the  510  rules  in  the  rule  base.  The 


selected  implementato  r  techniques  for  an  expen  systems  rule  base  influence  both  system 
maintainability  and  <  .rformance.  Based  on  their  development  and  maintenance  experiences, 
the  panel  identit-  j  a  critical  need  for  a  support  environment  that  includes  a  real-time 
browser  for  runtime  "peeks"  into  the  system  and  knowledge-base  maintenance  tools. 


Recommendations 

Software  engineering  is  a  challenge  for  the  AI  community  due  to  the  evolutionary 
nature  of  AI  applications.  Software  engineering  is  one  of  the  strengths  and  advantages 
offered  by  a  properly  managed  Ada  environment.  We,  the  AI  with  Ada  community,  should 
develop  processes  and  tools  that  support  the  "engineering"  of  AI  applications  with  Ada 
The  AIWG  should  add  a  section  to  the  AIWG's  annual  report  to  describe  software 
engineering  processes  for  AI  applications  and  assess  the  progress  that  has  been  made  in  the 
use  of  software  engineering  principles  to  develop  AI  applications.  The  AIWG  should 
develop  a  database  cataloging  software  engineering  processes,  tools,  and  applications  w  ith 
periodic  publication  of  this  information  in  Ada  Letters. 

In  order  to  manage  and  engineer  AI  with  Ada  projects,  the  AIWG  should  establish  a 
set  of  software  metrics  that  are  compatible  with  the  evolutionary  nature  of  AI  applications. 
Further,  the  AIWG  should  conduct  annual  surveys  to  determine  the  current  values  for  the 
software  metrics  so  that  this  information  can  be  used  by  the  Ada  community  to  manage  and 
engineer  AI  applications.  The  software  metrics  should  be  published  in  the  AIWG's  annual 
report  and  Ada  Letters. 

The  AIWG  should  become  actively  involved  with  standards  bodies  that  are 
developing  standards  and  other  guidance  that  impact  the  development  of  AI  applications  with 
Ada.  Of  course,  the  first  step  in  this  process  is  to  identify  the  AI  specific  requirements  that 
need  to  be  communicated  to  the  standards  bodies.  The  issues  and  problems  discussed  at  this 
workshop  should  be  matured  into  concrete  requirements  and  recommendations  for  formal 
submittal  to  the  appropriate  standards  bodies. 

Many  of  the  issues  and  problems  faced  by  the  AI  with  Ada  community  are  the  same 
problems  faced  by  all  AI  researchers  and  developers.  The  AIWG  should  work  closely  with 
the  AI  community  to  concentrate  our  combined  efforts  on  solving  our  common  problems 
rather  than  focusing  on  the  perceived  differences  between  the  AI  and  the  Ada  communities. 
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APPENDIX  A 


Welcome  Message  by  Jorge  L.  Diaz-Herrera 
Editor's  Notes 

Jorge  L.  Diaz-Herrera  is  the  Chair  of  the  ACM  SIGAda  AIWG.  Jorge  opened  the 
AIWG  Workshop  with  a  welcome  message  describing  the  background  and  focus  of  both  the 
AIWG  and  the  workshop.  In  his  welcome  message,  Jorge  addressed  several  key  issues  with 
Ada,  AI,  real-time  systems,  and  embedded  systems. 
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APPENDIX  B 


The  Data  Fusion  Technology  Demonstrator  System  (DFTDS)  Project 

Editor's  Notes 

This  article  is  a  treasure  trove  of  information  about  the  advantages  of  using  Ada  to 
implement  large-scale  real-time  Knowledge-Based  Systems  (KBSs).  The  United  Kingdom's 
Defence  Research  Agency  (DRA)  has  implemented  a  large-scale  real-time  KBS  for  ship 
borne  Command  and  Control.  DFTDS  has  been  implemented  with  220  KSLOC  of  which  50 
KSLOC  implements  KBSs  for  time-critical  functions  such  as  data  fusion  and  situation 
assessment.  DFTDS  is  a  true  KBS  in  Ada  application  as  the  rules  are  coded  directly  in  Ada 
to  achieve  the  run-time  efficiency  required  by  Command  and  Control  applications  which 
must  process  data  from  radar,  sonar,  navigation,  electronic  support  measures  (ESM),  and 
other  sensors.  At  the  present  time,  DFTDS  is  installed  on  the  Royal  Nava!  Frigate  HMS 
Marlborough  and  is  undergoing  sea  trials. 

This  article  discusses  the  lessons  learned  using  Ada  to  implement  DFTDS.  Dr.  Miles 
finds  "Ada  as  a  language  has  been  found  to  be  quite  simple  to  use  for  a  large  real-time  KBS 
and  in  some  respects,  notably  run-time  efficiency,  abstraction,  exception  handling,  strict 
typing,  has  distinct  advantages  over  AI  toolkits  and  expert  system  shells  for  engineered  real¬ 
time  applications."  Prototyping  to  assess  the  potential  of  implementing  time-critical 
functions  such  as  those  required  for  data  fusion,  situation  assessment,  planning,  and  reaction 
are  discussed  in  this  article.  Dr.  Miles  also  describes  the  DFTDS  Ada  shell  and  the  use  of  a 
rule  specification  template  with  a  semi-formal  rule  specification  language  to  ensure  a 
consistent  style  was  used  to  specify  the  hypotheses  and  rules. 


AI  Topics:  Blackboard  architecture,  expert  system,  data  fusion,  situation  analysis,  planning 

Domain  Area:  Command  and  control 

Language  Interfaces:  Unknown 

Project  Status:  Undergoing  sea  trials  and  evaluation 

Size  of  Ada  Source  Code:  220  KSLOC 

Number  of  Rules  in  Knowledge-Based  System:  510 

Design/Development  Methodology:  Iterative  prototyping 

Hardware  Platforms:  MicroVAX  3800 
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THE  DATA  FUSION  TECHNOLOGY  DEMONSTRATOR  SYSTEM  (DFTDS)  PROJECT, 
(presented  at  the  SIGAda  meeting  Ada/AI  workshop,  Seattle,  Washington,  June  1992) 

John  A  H  Miles 

DRA  Portsdown,  Portsmouth,  Hampshire,  UK. 


ABSTRACT 

A  large-scale  real-time  Knowledge-Based  System  (KBS)  has  been  implemented  entirely  in 
Ada.  The  application  is  Data  Fusion  for  shipbome  Command  and  Control  and  the  KBS  is 
interfaced  to  several  sensor  systems,  a  relational  database  and  provides  five  user  workstations. 

Ada  was  chosen  for  its  real-time  performance  and  its  system  engineering  features.  An  Ada 
Shell  using  the  Blackboard  Model  has  been  developed  and  a  methodology  for  knowledge 
specification  and  implementation  in  Ada  has  evolved;  these  are  described  in  the  paper. 

The  real-time  performance  of  the  KBS  has  met  its  targets  and  few  problems  have  been 
encountered  with  the  Ada  development  system  chosen.  Experience  with  Ada,  lessons  learned 
and  suggestions  for  Ada  improvements  for  KBS  are  included. 


L  INTRODUCTION 

This  paper  describes  a  demonstrator  project  which  resulted  from  a  research  programme  to  look 
at  the  application  of  AI,  particularly  KBS  techniques,  to  command  and  control.  Several 
laboratory  prototypes  were  developed  for  functions  such  as  Data  Fusion,  Situation 
Assessment,  Planning  and  Reactive  Resource  Allocation.  The  success  of  these  prototypes  and 
particularly  the  potential  of  those  addressing  time-critical  functions  has  led  to  a  large-scale  Data 
Fusion  Technology  Demonstrator  System  (DFTDS)  being  constructed  and  fitted  to  a  warship 
for  a  two-year  period  of  sea  trials  (reference  [1]).  The  objectives  of  this  programme  are: 

•  to  explore  the  use  of  KBS  and  other  new  technologies  for  providing  automated  support 
to  Data  Fusion  and  Situation  Assessment  functions 

*  to  reduce  the  risk  of  procuring  systems  which  use  these  technologies  by  examining  all 
stages  of  the  development,  setting -to- work,  performance  and  in-service  support  of  a 
large-scale  prototype 

Both  the  conventional  software  and  knowledge-based  modules  of  the  DFTDS  have  been 
successfully  developed  in  Ada.  All  the  modules  including  KBS  operate  in  real-time,  typically 
processing  100  messages  per  second.  These  features  make  the  DFTDS  a  valuable,  in  some 
respects  probably  unique,  source  of  data  from  a  practical  A da/KBS  application. 


2.  DFIDS  DESCRIPTION 

Figure  1  shows  a  block  diagram  of  the  main  DFTDS  modules.  The  Front-End  Processor 
(FEP)  receives  real-time  input  messages  from  all  the  ship's  sensor  systems,  from  data 
communication  links  and  from  operators  via  the  user  interface  module.  Its  main  purpose  is  to 
present  the  Data  Fusion  module  with  messages  in  a  suitable  format  for  knowledge-based 
processing.  The  module  provides  filtering,  re-formatting  and  recording  of  messages.  It  can 
also  operate  in  a  replay  mode  using  data  previously  recorded  or  simulated  scenario  data. 
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Figure  1  DFTDS  Block  Diagram  of  Modules 
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The  Data  Fusion  Module  (DFM)  is  the  First  and  currently  largest  KBS  module  in  the  DFTDS. 
It  is  structured  along  the  lines  of  a  Blackboard  System  with  rules,  knowledge  sources  and  a 
blackboard  data  structure  entirely  written  in  Ada.  The  purpose  of  the  module  is  to  correlate 
and  combine  data  from  all  available  sources  (sensors  and  manual  inputs)  to  form  a  fused 
tactical  picture.  It  allows  operators  access  to  and  interaction  with  each  stage  in  the  reasoning 
through  the  user  interface  module.  Some  of  the  reasoning  in  DFM  requires  access  to  the 
geographic  and  encyclopaedic  databases  held  in  the  Database  Module.  Further  details  of  DFM 
operation  are  given  later. 

The  Database  Module  (DBM)  provides  geographic  data  (coastlines,  navigation  features,  seabed 
features,  shore  features,  air-lanes,  etc.)  for  display  and  for  answering  on-line  queries  from 
DFM.  Similarly,  the  encyclopaedic  database  in  DBM,  contains  ship,  aircraft,  shore  base, 
equipment,  identity  data  etc.,  and  provides  query  facilities  for  both  DFM  and  for  operators  via 
the  user  interface  module. 

All  user  interaction  with  the  DFTDS  other  than  system  control  and  database  maintenance  is 
carried  out  through  the  User  Interface  Module  (UIM).  This  module  provides  a  WIMP 
(Windows,  Icons,  Menus,  Pointer)  style  interface  on  five  colour  graphics  terminals  and  a  form 
fill  style  interface  for  some  data  entry  on  two  ordinary  terminals.  The  windows  on  the 
graphics  terminals  give  visibility  of  all  stages  in  the  data  fusion  process  and  allow  the  user  to 
configure  the  display  in  any  desired  manner  to  suit  his  tasks. 

The  Situation  Assessment  prototype  (SAP)  is  a  second  KBS  module  written  in  Ada  using  a 
similar  framework  to  DFM.  Its  purpose  is  to  take  the  detailed  tactical  picture  produced  by 
DFM  and  by  applying  further  stages  of  reasoning  construct  a  concise  situation  display  showing 
the  main  groupings  of  objects  and  a  prioritised  threat  table.  All  user  interaction  with  this 
module  is  also  through  the  UIM  so  that  it  appears  as  one  system  to  the  user  even  though  it  uses 
distributed  KBSs.  SAP  will  be  enhanced  in  functionality  in  a  second  phase  of  the  project. 

All  the  DFTDS  application  modules  communicate  using  the  facilities  provided  by  a  System 
Support  Module  (SSM).  This  module,  part  of  which  resides  with  each  application  module, 
also  provides  control,  timing,  monitoring  and  fault  reporting. 
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The  overall  size  of  the  Ada  source  code  in  the  DFTDS  is  about  220,000  DSI  (Deliverable 
Source  Instructions).  Of  this  about  50,000  DSI  are  KBSs.  Other  sizing  metrics  for 
individual  modules  arc  as  follows: 


DEM  KBS: 

Input  Message  Types 

Number  of  Rules 

Number  _,f  Knowledge  Sources 

SAP  KBS: 

Number  of  Rules 

Number  of  Knowledge  Sources 

DBM  (Encyclopaedic  Database): 
Number  of  Tables 
Number  of  Stored  Procedures 
Number  of  MMI  Forms 
Overall  data  size 

DBM  (Geographic  Database): 

Number  of  Tables 
Number  of  Queries 
Overall  data  size 


UIM: 

Number  of  windows 
Number  of  window  overlays 


100 

510 

180 


90 

31 


244 

112 

68 

15  Mbytes 


12 

23 

350  Mbytes 


44 

18 


The  main  performance  criteria  for  the  DFTDS  are: 

Front-end  maximum  input  message  rate 
Maximum  message  rate  input  to  DFM 
Maximum  response  delay  in  any  module 
Database  queries 


180  messages/second 
100  messages/second 
1  second 
20  per  second 


4,BARpWAR£  and  SOFTWARE 

The  DFTDS  runs  on  a  network  of  five  DEC  Microvax*  3800  processors.  Each  main  module 
(i.e.  FEP,  DFM,  DBM,  UIM,  SAP)  is  allocated  to  a  separate  processor.  This  ensures  that 
there  is  no  contention  for  processing  resource  between  modules.  Ethernet  with  Decnet* 
software  has  been  used  for  inter-processor  communications.  This  has  proved  to  have 
considerable  overheads  when  used  for  high  volumes  of  short  messages  and  on  some  routes 
messages  have  had  to  be  blocked  into  larger  packets  to  achieve  required  throughput. 

The  DEC  VMS*  operating  system  and  the  DEC  Ada^  compiler  have  been  used  throughout. 
Apart  from  the  SSM  and  the  encyclopaedic  database  server  each  module  is  a  single  Ada 
program  which  runs  as  a  single  VMS  process.  DBM  is  written  in  Ada  but  uses  a  commercial 
Relational  DBMS  (Sybase^)  for  storing  the  encyclopaedic  data  and  performing  queries.  The 
Ada  part  of  DBM  interfaces  to  the  SQL  routines  using  the  Sybase  provided  Ada  libraries. 


1  Microvax,  Decnet,  VMS,  Dec  Ada  are  trademarks  of  Digital  Equipment  Corporation. 

2  Sybase  is  a  trademark  of  Sybase  inc. 
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5.  CHOICE  OF  ADA 

Previous  laboratory  prototypes  of  the  data  fusion  function  had  used  a  specially  written 
blackboard  framework,  called  MXA  (reference  (2]>,  because  no  suitable  real-time  KBS 
package  could  be  found  at  that  time  (1983);  MXA  was  based  on  the  Pascal  language.  This 
framework  provided  some  elaborate  set  intersection  mechanisms  but  would  only  cope  with 
simple  scenarios  in  real-time.  At  the  beginning  of  the  DFTDS  programme  a  method  had  to 
found  to  achieve  real-time  performance  with  the  data  rates  found  in  a  real  operational  setting. 

A  survey  of  available  tools  was  carried  out  and  three  styles  of  implementation  were 
benchmarked  using  a  subset  of  a  data  fusion  knowledge  base  with  a  test  scenario.  This 
benchmarking  experiment  is  more  fully  reported  and  discussed  in  references  [3]  &  [4], 
Briefly  it  showed  that  a  'pure'  production  system  implemented  in  a  LISP  machine  environment 
was  far  too  slow  whereas  a  hybrid  production  system  -  procedural  language  based  solution 
using  an  AI  toolkit  specially  designed  for  embedded  real-time  applications  produced  better  than 
real-time  performance.  Even  though  good  performance  was  achieved  with  this  toolkit,  only  a 
conventional  language  solution  using  a  blackboard  style  framework  actually  achieved  the 
required  real-time  performance.  Figure  2  summarises  the  benchmark  implementations  and 
results. 


System 

Hardware 

Time  in  minutes  to  process 
30  minutes  of  data 

Fraction  of 

real-time 

ART 

Symbolics  LISP  machine 

600 

20.0 

MUSE 

Sun  4/260  workstation 

15 

0.5 

Ada  Shell 

Microvax  3500  workstation 
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Figure  2  Benchmark  Implementations  and  Results 


Ada  was  chosen  for  the  conventional  language  implementation  because  it  is  the  latest  language 
specifically  designed  for  real-time  applications  and  on  the  results  of  some  very  simple 
benchmarks  seemed  to  have  no  significant  performance  penalties  providing  a  good  compiler 
was  chosen.  Various  compiler/machine  combinations  were  tested  using  the  data  fusion 
benchmark  and  the  DEC  Ada  compiler  running  on  a  microvax  proved  to  be  the  best 
combination  for  speed,  support  and  reliability.  These  results  were  obtained  in  the  first  quarter 
of  1988. 


fL  KBS  IMPLEMENTATION.  VSCSQ  ADA 

In  order  to  produce  the  Ada  data  fusion  benchmark  a  blackboard  style  framework  had  to  be 
developed.  This  proved  relatively  simple  for  a  small  rule-set  but  it  was  recognised  that  a  more 
engineered  version  with  diagnostic  facilities  would  be  required  to  construct  a  large  knowledge 
base  and  an  'Ada  Shell'  was  produced.  Initially  an  Ada  tasking  approach  was  taken  to  various 
functions  of  this  shell  but  this  was  found  to  introduce  run-time  overheads  owing  to  the  high 
input  message  rates  required.  The  final  version  has  proved  very  efficient  and  has  remained 
largely  unchanged  over  the  two  and  a  half  year  DFTDS  development  programme. 

A  particular  attribute  of  the  Ada  KBS  approach  taken  is  that  the  rules  are  coded  directly  in  Ada 
making  it  a  true  KBS  in  Ada  application.  Apart  from  the  excellent  run-time  efficiency  that  this 
approach  gives  it  has  also  been  found  that  the  abstraction  capability  of  Ada  and  the  flexibility  of 
a  conventional  language  used  in  conjunction  with  a  blackboard  model,  allows  for  an  easier  and 
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better  designed  implementation  than  the  more  common  production  system  approach.  The  latter 
is  driven  by  the  need  for  exact  patterns  to  be  specified  for  the  implicit  rule  firing  mechanism  to 
work  whereas  the  rule  firing  mechanism  in  the  Ada  blackboard  approach,  although  requiring 
more  hand-crafting,  is  much  simpler,  flexible  and  efficient  and  results  in  more  concise  rules. 

A  drawback  in  using  Ada  for  the  DFM  is  that  a  KBS  demands  a  large  amount  of  global  data  in 
the  form  of  blackboard  hypotheses,  whereas  Ada  data  handling  concentrates  on  data  abstraction 
and  data  hiding.  The  Ada  compilation  suite  is  not  best  suited  to  deal  with  this  global  data  and 
this  means  that  relatively  minor  changes  to  the  blackboard  specification  result  in  prohibitively 
long  re-compilation  times  -  lasting  an  entire  working  day  in  some  cases. 


IJ&AStiEU* 

Figure  3  shows  a  diagram  of  the  components  of  the  Ada  shell  and  the  blackboard  elements  that 
are  part  of  the  application.  In  normal  operation  the  Scheduler  takes  the  highest  priority  Event 
off  the  Event  Queue  and  calls  the  appropriate  Knowledge  Source(s)  (if  any).  On  completion 
of  the  Knowledge  Source  control  returns  to  the  Scheduler  and  the  cycle  repeats.  The 
Scheduler  also  checks  for  input  messages  and  for  delayed  Events  as  part  of  its  cycle.  A  Clock 
task  maintains  time  and  an  Input  task  places  input  events  on  the  Event  Queues. 
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Figure  3  Ada  Shell  Components 
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For  development  purposes  there  is  a  Browser  task  which  can  be  invoked  by  typing  control-C 
on  the  Browser  Terminal.  This  action  transfers  control  to  the  Browser  task  and  stops  time  and 
further  events  from  being  processed.  The  Browser  has  commands  for  displaying  the  contents 
of  the  Blackboard  and  Event  Queues,  for  setting  breaks  on  time/events  and  for  stepping  single 
events.  It  also  allows  events  on  the  queues  iu  be  explicitly  fired  in  any  order,  events  to  be 
removed  and  specific  events  to  be  inhibited  or  permitted.  A  DCL  sub-process  can  be  spawned 
from  the  Browser  to  provide  access  to  the  host  operating  system  for  viewing  files,  etc.  With 
these  simple  facilities  the  action  of  each  Knowledge  Source  can  be  closely  monitored. 
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A  blackboard  system  has  a  number  of  components  and  these  are  represented  in  Ada  in  the 
following  manner 


The  Blackboard  -  this  is  a  data  structure  consisting  of  hypotheses  and  links.  Hypotheses  are 
implemented  as  record  structures  and  these  can  be  easily  specified  in  Ada  Links  are  provided 
by  set  operations  in  a  generic  package.  These  operations  allow  links  with  a  many-to-one,  one- 
to-many  and  many-to-many  relationships  to  be  represented. 

Rules  -  these  are  written  as  if...then..endif  constructs  within  a  procedure.  The  use  of  a 
procedure  allows  rules  to  be  called  from  one  or  more  knowledge  sources.  Although  in 
principle  similar  to  production  rules,  the  Ada  rules  tend  to  contain  more  functionality  and  have 
more  complex  actions  than  those  typically  found  in  other  KBSs.  Output,  in  the  form  of 
messages  to  other  modules,  is  generated  directly  by  the  rules.  This  output  includes  copies  of 
blackboard  data  and  explanations  generated  on  request  from  the  UIM. 

Knowledge  Sources  -  these  are  implemented  as  procedures  which  generally  contain  nothing  but 
calls  to  rule  procedures.  In  some  cases  it  has  proved  convenient  to  provide  some  logic  within 
knowledge  sources  for  more  'intelligent'  invokation  of  rules. 

Events  -  there  are  three  types  of  event;  input  events  which  have  a  type  and  an  input  message 
associated  with  them,  normal  events  which  have  a  type  and  a  hypothesis  reference  associated 
with  them  and,  delayed  t  ents  which  have  a  type,  a  hypothesis  reference  and  a  time  associated 
with  them. 


The  Ada  shell  operates  as  follows: 

An  input  event  is  generated  for  every  input  message  and  there  is  a  one-to-one  correspondence 
between  input  event  types  and  message  types.  Other  events  are  generated  explicitly  by  rules  to 
indicate  particular  changes  to  the  blackboard  or  to  invoke  processing  at  a  later  time.  In  theory 
there  could  be  an  event  type  for  every  type  of  change  to  every  type  of  hypothesis  but  in  practice 
events  types  are  only  created  as  required  to  suit  the  knowledge  source  and  rule  structure. 

There  are  four  normal  event  queues  of  different  priority  and  the  delayed  event  queue.  The 
scheduler  takes  the  highest  priority  event  and  using  a  look-up  table  calls  the  knowledge  source 
applicable  to  the  type  of  event.  There  can  be  more  than  one  knowledge  source  assigned  to  an 
event  or  none  at  all;  in  practice  there  is  usually  just  one  since  any  rules  that  can  potentially  fire 
for  the  event  can  be  put  in  a  single  knowledge  source. 

The  event  contains  either  a  message  or  a  hypothesis  reference  to  which  all  of  the  rules  called  by 
the  knowledge  source  have  access.  In  other  words  the  rules  are  directed  at  the  change  in  the 
blackboard  and  they  then  apply  their  exact  conditions  to  attempt  to  fire.  This  mechanism 
means  that  there  is  no  guarantee  that  a  rule  called  by  a  knowledge  source  will  fire  and 
consequently  some  time  may  be  wasted  evaluating  rule  conditions  that  fail  but  the  event 
mechanism  is  so  simple  and  efficient  that  it  easily  out-performs  more  rigorous  approaches  such 
as  the  RJETE  algorithm  used  in  productions  systems. 

Rules  from  different  functions  can  be  called  from  the  same  knowledge  source  if  their  firing 
conditions  are  similar  thus  implementing  the  opportunistic  nature  of  the  blackboard  model.  It 
would  be  possible  to  implement  more  complex  scheduling  but  the  simple  system  described  has 
been  found  quite  adequate  for  the  real-time  data  fusion  application.  It  has  sometimes  proved 
useful  to  invoke  a  rule  conditionally  depending  on  the  result  of  a  previous  rule  but  this  can 
easily  be  implemented  within  the  knowledge  source. 
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The  knowledge  base  is  specified  in  terms  of  Hypotheses  and  Rules;  definition  of  events  and 
knowledge  sources  is  considered  to  be  part  of  the  implementation  process.  Rules  arc  written 
in  a  declarative  style  so  that  each  one  completely  defines  when  and  what  processing  will  occur. 
A  specification  template  has  been  designed  (figure  4)  together  with  a  semi-formal  language  to 
ensure  a  consistent  style  of  specification.  The  semi-formal  language  contains  a  range  of 
keywords  including  'if,  'and',  'actions’,  and  a  number  of  verbs  for  various  types  of  action. 
Each  rule  is  restricted  at  the  top  level  to  one  or  more  'anded'  conditions  followed  by  one  or 
more  actions.  Conditions  and  actions  may  invoke  further  operations  which  can  be  of  arbitrary 
complexity.  These  operations  are  specified  with  the  rules  in  a  pseudo-code  form  because  they 
contain  important  domain  knowledge.  In  other  cases  the  rule  specification  may  use  a  function 
or  procedure  in  order  to  keep  the  specification  concise  but  for  which  the  details  of  operation  arc 
either  self-explanatory  or  of  a  house-keeping  nature  unrelated  to  the  domain  knowledge;  in 
these  cases  no  further  specification  is  given. 


Rule  No  RXXXnnn  <name> 
(a'  r  mmary 
(\  -vdle  Specification 
kuLE'RHWnnn  <name> 
if  <condition  1> 
and  <condition  2> 

actions 


END  RULE  RXXXnnn 

(c)  Assumptions/Limitations 

(d)  Definition  of  Criteria 

(e)  Supporting  Operations 

(f)  Hypotheses 

(g)  Data  Structures 


Figure  4  Rule  Specification  Template 


The  summary  (a)  of  each  rule  is  an  English  version  of  the  domain  knowledge  contained  by  the 
rule.  The  rule  specification  (b)  is  a  more  precise  definition  of  what  the  rule  does  and  it  is 
required  that  the  Ada  code,  although  more  detailed,  follows  the  content  and  order  of  this 
specification,  thus  retaining  the  KBS  philosophy  of  keeping  specification  and  source  code 
closely  aligned.  Part  (c)  is  a  comment  heading  under  which  any  assumptions  or  limitations 
about  the  scope  of  the  rule  can  be  noted.  Part  (d)  provides  a  'definition  of  criteria'.  This  is 
used  to  reference  more  detailed  specifications  of  any  conditions  used  in  the  rule;  Complex 
conditions  or  simply  constants  are  generally  expressed  in  meaningful  words  within  the  rule 
specification  in  order  to  keep  it  concise.  Part  (e)  is  intended  for  specifying  any  supporting 
mathematical  operations  used  by  the  rule  -  in  practice  this  is  rarely  used  because  such 
operations  are  usually  not  visible  at  the  rule  specification  level  and  if  they  are,  they  arc  mostly 
self  explanatory.  Parts  (0  and  (g)  list  hypotheses  and  other  blackboard  data  structures 
accessed  by  the  rule.  This  was  originally  intended  as  a  means  to  trace  rule  access  and  hence 
interaction  between  rules  but  such  analysis  has  still  to  be  performed. 

Specification  of  the  knowledge  base,  principally  in  terms  of  hypotheses,  rules  and  operations, 
is  contained  in  a  document  called  the  'Acquired  Knowledge-base  Specification'  or  AKS'. 
This  document  is  then  used  both  for  implementation  in  Ada  and  for  validation  of  the 
knowledge-base  by  domain  experts  -  it  is  the  definitive  reference  for  users,  implemented  and 
maintainers.  Because  the  user  interface  to  the  KBS  is  also  a  very  complex  piece  of  software,  a 
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separate  specification  has  been  generated  to  detail  every  window  and  interaction  with  the  user 
and  knowledge-base. 


9.  PFTPS  PROJECT  STATUS 

The  main  development  phase  of  the  DFI  DS  is  complete  and  the  system  including  the  data 
fusion  KBS  has  been  integrated  and  set-to-work.  Current  activity  is  concerned  with  working- 
up  the  system  onboard  the  trials  ship  and  tuning  the  knowledge-base  to  optimise  its 
performance  against  real  data  from  ship's  sensors. 


A  trials  and  evaluation  programme  has  been  formulated  and  has  already  commenced.  It 
encompasses  die  following  areas  of  investigation: 

•  KBS  Performance 

•  Technology  Issues 

•  Human  Computer  Interface  (HCI) 

•  Manning  and  Personnel 

•  Value  to  command 

KBS  Performance  involves  the  measurement  of  data  fusion  performance  under  a  variety  of 
normal  and  extreme  operating  conditions.  This  performance  can  then  be  compared  with  that  of 
existing  more  manual  systems,  and  the  value  and  limitations  of  the  KBS  approach  can  be 
found. 

Technology  Issues  include  all  technical  aspects  of  development  and  in-service  support;  the  use 
of  Ada  is  one  such  topic. 

The  Human  Computer  Interface  (HCI)  has  novel  features  for  this  type  of  shipbome  system 
both  in  terms  of  its  WIMP  style  and  its  KBS  features  such  as  explanations.  Feedback  from 
users  who  will  be  using  the  system  under  realistic  conditions  will  be  sought  and  will  form 
valuable  guidelines  for  future  KBS  procurements. 

Manning  and  Personnel  issues  will  be  examined  to  determine  the  impact  which  the  advanced 
level  of  automated  support  enabled  by  KBS  technology  has  on  the  level  of  manning,  the  skills 
and  knowledge  required  and  the  training  requirements. 

Value  to  Command  addresses  the  impact  which  the  quality  of  output  from  the  DFTDS  will  have 
on  the  higher  levels  of  command  within  the  ship.  If  the  DFTDS  produces  a  more  complete, 
accurate  and  timely  tactical  picture  and  assessment  then  this  should  result  in  improvements  to 
the  decision  making  process  and  hence  to  the  whole  command  and  control  performance. 


10.  LESSONS  LEARNED 

At  this  stage  in  the  programme  the  main  lessons  learned  concern  the  development  process. 
Those  that  relate  to  the  use  of  Ada  for  KBS  applications  are  as  follows: 

Knowledge  Engineering  -  Ada  as  a  language  has  been  found  to  be  quite  simple  to  use  for  a 
large  real-time  KBS  and  in  some  respects,  notably  run-time  efficiency,  abstraction,  exception 
handling,  strict  typing,  has  distinct  advantages  over  AI  toolkits  and  expert  system  shells  for 
engineered  real-time  applications.  The  drawbacks  to  Ada  are  that  does  not  directly  provide  the 
KBS  styles  of  programming  and  these  have  to  be  hand-crafted  before  knowledge  engineering 
can  start.  It  might  be  possible,  of  course,  to  design  a  number  of  general  purpose  packages  to 
suit  a  range  of  Ada  KBS  applications. 
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DEC  Vax  Ada  Experience  -  this  Ada  system  has  proved  extremely  reliable  with  very  few  bugs 
or  limitations  being  experienced  throughout  the  DFIDS  development.  It  does,  however, 
require  substantial  computing  resources  to  support  the  development  teams  and  as  the  software 
has  grown,  the  re-compile  times  have  become  very  long  (several  hours).  This  seems  to  be  due 
to  contention  for  access  to  central  libraries.  No  doubt  other  Ada  systems  have  similar 
problems  with  large  systems.  It  is  a  particular  difficulty  with  a  KBS  as  a  considerable  amount 
of  iteration  in  developing  and  tuning  the  knowledge  base  is  inevitable  and  indeed  part  of  the 
development  philosophy. 

Performance  -  The  run-time  performance  targets  have  been  achieved  though  some  optimisation 
has  been  carried  out;  the  use  of  a  run-time  analysis  tool  (PCA)  has  proved  very  useful  for 
identifying  where  most  time  is  spent.  Use  of  generics  seemed  to  cause  unexpected  overheads 
and  in  some  cases  these  have  been  removed.  Run-time  checking  and  exception  handling  has 
proven  to  be  most  useful  in  ensuring  that  the  system  continues  running  even  when  errors 
occur,  the  overheads  in  this  checking  seem  surprisingly  low. 

Ada  Shell  Facilities  -  Although  rudimentary,  the  facilities  of  the  Ada  shell  have  proved  to  be 
extremely  useful  for  developing  the  data  fusion  KBS.  The  small  units  of  processing 
(knowledge  sources)  and  simple  scheduling  cycle  coupled  with  the  break,  step  and  display 
facilities  of  the  Browser  provide  good  visibility  of  the  program  operation,  though  it  is  intrusive 
and  non-real-time.  In  fact  the  facilities  would  be  beneficial  to  any  Ada  program  whether 
considered  a  KBS  or  not  Of  course  the  facilities  are  not  as  sophisticated  as  an  AI  toolkit 
because  the  Ada  language  does  not  allow  easy  access  to  the  program  source  code  at  run-time 
(though  it  can  be  used  in  conjunction  with  the  source  level  Ada  debug  facility).  Also,  the 
display  of  blackboard  data  structures  by  the  Browser  has  to  be  hand-crafted  as  the  system  is 
developed.  If  the  Ada  shell  was  integrated  with  the  Ada  debug  tool  then  these  drawbacks 
could  be  overcome. 

Ada  Language  Facilities  -  Ada  is  obviously  not  an  AI  prototyping  language  but  as  a  means  of 
engineering  a  large  real-time  KBS  application  it  has  proved  quite  effective.  A  few  features 
could  be  added  to  make  AI  programming  easier,  such  as: 

•  Variables  could  have  a  'nil'  state;  KBSs  particularly  deal  with  partial  data  sets  and  very 
large  numbers  of  variables.  To  indicate  whether  a  variable  is  set  in  Ada  a  further 
variable  must  be  defined  whereas  in  most  AI  languages  a  variable  can  have  a  nil'  state 
to  indicate  it  is  unset 

•  Objects;  the  DFTDS  has  used  a  rule-based  approach  throughout  because  rules  are 
simple  and  reasonably  predictable  in  processing  terms  which  is  important  in  a  real-time 
system.  However,  it  is  recognised  that  Object  Oriented  Programme  (OOP)  is  useful 
for  some  knowledge  representation  and  has  been  used  in  some  of  our  laboratory 
prototypes.  Explicit  support  for  OOP  would  be  a  useful  addition  to  Ada. 

•  Type  Hierarchies;  Ada  has  a  rather  inflexible  system  of  types,  sub-types  and 
discriminents.  It  should  be  possible  to  define  a  hierarchical  data  structure  and  to  wntc 
packages  or  procedures  with  any  required  visibility  of  that  structure.  Currently,  Ada 
requires  code  to  have  full  downwards  visibility  of  the  structure  even  though  it  may  only 
need  visibility  of  one  level.  An  example  of  this  is  where  software  is  required  to  handle 
messages  without  needing  to  know  their  content.  A  similar  problem  arose  in  the 
design  of  the  blackboard  hypothesis  structure  where  there  are  a  number  of  different 
types  of  hypothesis  but  general  purpose  links  are  required  between  them.  Eventually  a 
single  type  of  hypothesis  was  defined  with  discriminents  for  the  contents. 

•  Atoms;  For  symbolic  processing  most  AI  languages  have  atoms'  which  are  general 
purpose  identifiers  for  arbitrarily  complex  structures.  Ada  has  enumerated  type 
variables,  all  possible  values  for  which  must  be  specified  at  compile  time,  and  strings 
Strings  are  cumbersome  in  a  strictly  typed  language  and  are  inefficient  for  processing 


purposes.  The  DFTDS  implements  an  Ada  interface  to  a  relational  database;  the 
database  holds  identifiers  as  strings  and  the  interface  converts  some  of  these  into 
enumerated  types,  where  they  match  types  in  the  rest  of  the  system,  and  some  it  keeps 
as  strings.  The  problem  is  that  the  strings  are  unchecked  and  inefficient  and  the 
enumerated  types  are  inflexible  so  that  simple  changes  to  the  database  require  changes 
throughout  the  Ada  code.  There  seems  to  be  a  need  for  a  run-time  enumerated  type 
which  is  held  internally  as  a  number  but  appears  as  a  symbol  at  input  and  output 


11.  FUTURE  WORK 

As  far  as  the  DFTDS  trials  and  evaluation  programme  is  concerned  the  use  of  the  Ada 
language,  Ada  environment  and  tools  will  be  reported  under  the  technology  issues  area. 
Further  studies  are  planned  into  the  knowledge  engineering  and  knowledge  base  maintenance 
aspects  of  the  DFTDS.  These  studies  will  formalise  the  methodology  for  using  Ada  for  KBS 
which  has  evolved  and  will  investigate  tools  for  assisting  the  maintenance  of  a  large  knowledge 
base.  Now  that  the  DFTDS  rule  set  contains  500  or  more  rules  it  is  becoming  increasingly 
difficult  to  maintain  the  specification  and  carry  out  verification  of  changes. 


12.  CONCLUSIONS 

The  DFTDS  project  has  proved  that  Ada  can  be  used  effectively  for  large  real-time  KBS 
applications  with  very  little  additional  support  software. 

A  large  knowledge  base  has  been  established  and  this  is  entering  an  evaluation  phase  which 
will  report  on  all  aspects  of  performance,  procurement,  support,  HCI,  manning  and  value  to 
users. 

The  experience  with  the  Ada  products  selected  has  been  good  and  some  suggestions  for 
improvement  to  Ada  for  KBS  have  been  put  forward. 
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APPENDIX  C 


Validation  and  Verification  of  Safety  Critical  Knowledge- Based  Systems 


Editor's  Notes 


The  United  Kingdom's  Defence  Research  Agency  (DRA)  has  a  three  year  program  to 
study  the  validation  and  verification  (V&V)  of  safety  critical  Knowledge-Based  Systems 
(KBSs)  with  the  use  of  domain  dependent  virtual  machines.  In  the  first  year,  a  Data  Fusion 
Language  (DFL)  was  identified  and  formally  defined  and  the  requirements  for  a  virtual 
machine  to  support  the  DFL  were  established.  A  virtual  machine  to  support  the  DFL  was 
designed  and  the  design  verified  in  the  second  year.  A  virtual  machine  prototype  to  support 
the  DFL  will  be  implemented  in  the  third  year.  At  the  present  time  (year  two),  the  virtual 
machine  design  is  being  verified. 

This  article  provides  some  useful  insight  into  the  potential  of  domain  dependent 
virtual  machines  for  V&V  of  large-scale  Ada  KBSs.  This  approach  is  being  tested  with  a 
subset  of  the  Ada  KBS  which  is  currently  installed  on  the  Royal  Naval  Frigate  HMS 
Marlborough  and  described  in  the  associated  article  "The  Data  Fusion  Technology 
Demonstrator  System  (DFTDS)  Project". 
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Validation  and  Verification 


This  presentation  concerned  an  investigation  of  the  use  of  domain  dependent  virtual  machines 
to  aid  the  verification  and  validation  processes  carried  out  during  the  development  process  of  a 
Knowledge-Based  Systems  (KBS).  This  virtual  machine  approach  to  KBS  development  may  be 
summarised  as:  prototype  KBS  for  solving  one  or  more  specific  problems  from  a  domain  of 
interest;  identify,  from  this  prototyping  exercise,  a  domain  specific  language  (DSL)  for  expressing 
solutions  to  problems  in  this  domain;  implement  a  virtual  machine  that  has  this  DSL  as  its  source 
language;  use  this  virtual  machine  to  implement  prototypes  and  deliverable  KBS.  The  potential 
advantages  of  this  approach  include: 

•  much  of  the  development  process  is  brought  into  the  realm  of  conventional  software 
engineering; 

•  less  distortion  of  domain  knowledge  than  would  arise  if  it  was  expressed  in  the  knowledge 
representation  language  (KRL)  of  a  development  environment  or  KBS  shell; 

•  satisfying  engineering  constraints,  e.g.  required  speed  of  execution,  target  hardware,  etc., 
can  be  addressed  within  virtual  machine  design,  independently  of  particular  knowledge 
representations. 

Additional  benefits  may  result  if  formal  semantics  are  defined  for  the  domain  specific  language 
and  formal  specification  and  development  techniques  are  used  to  ensure  that  the  virtual  machine 
implements  those  semantics.The  utility  of  a  virtual  machine  approach  is  to  be  tested  by  the  creation 
of  a  Data  Fusion  KBS,  using  a  suitable  subset  of  the  knowledge-base  of  a  large  scale  Knowledge- 
Based  Sensor  Data  Fusion  Technology  Demonstrator  System  currently  installed  on  a  Royal  Naval 
Frigate  HMS  Marlborough. 

The  work  began  in  1st  June  1991,  just  over  a  year  before  this  presentation  and  most  of  the 
work  is  being  carried  out  at  the  Royal  Military  College  of  Science,  Shrivenham,  Swindon, 
Wiltshire,  England.  We  are  now  at  the  stage  of  verifying  our  virtual  machine  design.  By  June 
1993  we  will  have  completed  a  review  into  the  first  two  years  of  work.  This  review  is  intended  to 
assess  the  viability  of  our  original  concept  and  perhaps  redirect  the  final  year  of  the  investiganon  in 
the  light  of  the  problems  we  have  encountered. 


Separate  out  those  aspects  of  the  system 


that  can  be  developed  using  conventional  software 

engineering  techniques 

from 

those  that  are  particular  to  the  knowledge-base 

approach. 


This  research  item  investigates  the  effectiveness  of 
formally  engineering  virtual  machines  to  support 
application  specific  knowledge  representation 
languages.  The  objectives  of  this  item  are: 

•  to  being  to  bring  as  much  possible  of  the 
development  of  KBS  within  the  domain  of 
conventional  software  engineering; 

•  to  apply  the  software  engineering  techniques 
associated  with  ‘Safety  Critical’  software  to 
appropriate  parts  of  KBS  development. 

•  Thus  to  increase  the  effectiveness  of  verification 
and  validation  of  such  Knowledge-Based  Systems. 


Verification  and  Validation 


Verification 


•  The  use  of  formal  specification  techniques  in  the 
design  and  implementation  of  the  virtual  machine. 

•  The  formal  definition  of  the  semantics  of  the 
supported  application  specific  KRL. 

Validation 


The  proof  of  properties  of  a  system  implemented  in 
the  application  specific  KRL  that  is  simpler  to 
perform  than  the  direct  formal  transformation  of  a 
knowledge  specification  into  an  implementation. 


Representation  Languages  (KRLs) 


•  The  success  of  the  virtual  machine  approach  rests 
with  their  definition. 

•  They  will  imply  one  or  more  relevant  knowledge- 
based  system  paradigms 

•  They  will  incorporate  domain  specific  abstractions. 
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Naval  Data  Fusion  Example 


TDS  Like  Application 
Naval  Data  Fusion  Language 
Data  Fusion  Blackboard  System  Primitives 
Blackboard  System  Primitives 
Ada  Code 


“Validation  &  Verification 
of  ‘safety  critical’  KBS” 

Is  being  done  under  a  3  year  contract  by  the  Royal 
Military  College  of  Science  at  Shrivenham. 

The  work  commenced  on  the  1st  June  1991  and  is, 
therefore,  due  for  completion  by  the  end  of  May 
1994. 
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Year  1: 

to  identify  and  formally  define  a  Data  Fusion  Language  for 
Naval  Tactical  Data  Fusion,  and  to  establish  the  requirements 
for  a  virtual  machine  to  support  this  Data  Fusion  Language. 

Year  2: 

to  produce  a  verified  design,  using  appropriate  structured  and 
formal  techniques,  for  the  virtual  machine  to  support  the  Data 
Fusion  Language,  and  to  animate  the  design  to  validate  the  Data 
Fusion  Language. 

Year  3: 

to  implement  a  prototype  virtual  machine  to  support  the  Data 
Fusion  Language,  and  to  evaluate  its  utility  for  Naval  Tactical 
Data  Fusion  and  the  ability  of  the  approach  to  provide  high 
levels  os  assurance  for  knowledge-based  systems  for  domains 
where  reliability  is  regarded  as  a  key  requirement  of  those 
systems. 


First  Year 

(June  1991-June  1992) 

V  Familiarisation  -  with  the  ARE  Technical  Demonstrator  Programme 
and  Data  Fusion  Technical  Demonstrator  System. 

V  Review  Formalisms  -  vdm;  z;  [b;j  csp  r  ccs,*  3  obj  [  and 

similar  methods  3  and  Scott«Strachey  denotational  semantics. 

V  Specify  Subset  of  TDS  Knowledge-Base  -  delivered  in 

English,  VDM  and  Ada  form. 

V  Identify  Data  Fusion  Language  •  postulated  in  terms  of  a 

BNF  syntax. 

->  Define  Formal  Semantics  of  the  Data  Fusion 

Language  •  taking  slightly  longer  than  anticipated  but  a  complete 
definition  will  be  provided  -  almost  complete. 

X  Investigate  Non-functional  Requirements  -  postponed 

after  an  initial  look. 
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Definition  of  Data  Fusion  Language 
Formal  Semantics 


•  No  one  formalism  sufficient  for  defining  the  DFL 
semantics. 

•  VDM  for  the  semantics  of  the  parts  of  the  DFL 
used  to  define  the  blackboard  structure. 

•  CSP  for  the  semantics  of  the  parts  of  the  DFL  used 
to  define  the  control  structures. 

•  Validity  of  the  particular  VDM/CSP  mix  will  be 
demonstrated. 
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Not  fully  investigated  because: 

•  Priority  given  to  the  definition  of  DFL  semantics; 

•  Available  time  limited; 

•  Initial  investigation  indicated  problem  even  more 
difficult  than  originally  seemed. 


Second  Year 

Identify  Virtual  Machine  Design  Approach .  an 

investigation  of  structured  and  formal  methods  to  see  which  are  applicable 
and  the  tool  support  which  is  available. 

Design  Virtual  Machine  •  design  will  comprise  an  architectural 
model  and  a  formal  specification  defining  that  model. 

Verify  Virtual  Machine  Design  *  it  is  unlikely  that  fun 

formal  verification  can  be  brought  to  a  logical  conclusion  within  the  cost 
constraints  of  the  project.  However  it  is  hoped  that  sufficient  progress  will 
be  made  to  establish  what  may  be  achievable  in  this  area. 

Animate  Virtual  Machine  Design  -  by  translating  the  formal 

specification  for  the  virtual  design  into  an  executable  language. 


Review  Data  Fusion  Language  •  so  that  amendments  to  its 

definition  can  be  made  in  the  light  of  the  design,  design  validation  and 
design  animation  of  the  virtual  machine. 


Third  Year 

Identify  Virtual  Machine  Implementation  -  a  review  of 

languages  and  tools  suitable  for  the  implementation  of  the  virtual  machine 
on  an  available  hardware  such  as  a  Vax  or  Sun. 

Implement  Virtual  Machine  -  testing  win  be  based  upon 

conformance  to  the  formal  semantics  of  the  Data  Fusion  Language  and  the 
virtual  machine  design  (this  will  be  the  largest  single  task  of  the  project). 

Evaluate  Virtual  Machine  -  In  order  to  evaluate  the  utility  of 
the  virtual  machine  approach,  a  different  subset  of  the  TDS  knowledge¬ 
base  will  be  identified  and  translated  into  the  Data  Fusion  Language  -  a 
final  report  on  the  utility  of  this  virtual  machine  approach  will  be  generated 
bringing  this  contract  to  a  close. 
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End  Products 

•  The  Virtual  Machine  Prototype. 

•  Assessment  Of  The  Virtual  Machine 
Approach. 
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APPENDIX  D 


Embedded  Real-Time  Reasoning  Concepts  and  Applications 


Editor’s  Notes 

Boeing  has  successfully  implemented  several  intelligent  real-time  embedded  avionics 
systems  with  the  Ada  programming  language.  These  systems  include  a  Tactical  Cockpit 
Mission  Manager  (40  KSLOC  of  Ada,  250  rules),  a  Search  Area  Planner  Tool  (45  KSLOC  of 
Ada,  300  rules),  and  an  Automated  Sensor  Manager  (153  KSLOC  of  Ada,  1,181  rules).  This 
information  is  in  these  proceedings,  but  1  wanted  to  emphasize  that  these  applications  are 
operational  AI  with  Ada  applications  totaling  of  238  KSLOC  that  implements  1 ,73 1  rules  of 
complex  reasoning  based  avionics  systems. 

Boeing  developed  these  applications  with  an  innovative  approach  that  combines  the 
"engineering"  rigor  of  conventional  software  development  with  Al  rapid  prototyping 
techniques.  Ada  Real-Time  Inference  Engine  (ARTIE)  is  a  tool  which  provides  an 
interpretive  development  environment  and  a  small,  fast  embeddable  inference  engine  for 
runtime  performance.  ARTIE  offers  an  innovative  approach  to  rapidly  prototyping 
embedded  real-time  reasoning  software  with  an  interpretative  development  mode  and  an 
automatic  code  generator  for  generating  embedded  real-time  code  for  execution  on  the  target 
platform.  Workshop  participants  were  in  mutual  agreement  that  interpretive  tools  such  as 
ARTIE  are  a  valuable  asset  for  real-time  embedded  systems  prototyping,  development,  and 
debugging. 

AI  Topics:  Cooperating  expert  systems,  forward  chaining,  iteratively  recursive  inferencing 
Domain  Area:  Intelligent  Avionics  Systems 

Language  Interfaces:  A  Pascal  inference  engine  was  developed  in  1987. 

Project  Status:  Test  and  evaluation 

Size  of  Ada  Source  Code:  ARTIE  is  25  KSLOC  with  associated  tools  and  the 

embedded  inference  engine  is  less  than  3  KSLOC; 

Avionics  reasoning  based  applications  total  238  KSLOC 

Number  of  Rules  in  Knowledge-Based  System(s):  1,731  rules  in  3  Ada  applications 

Design/Development  Methodology:  Iterative  prototyping 

Hardware  Platforms:  Apollo  3000, 4000,  and  590;  Sun  2/60,  3/60,  and  4/60;  Digital 

Equipment  MicroVAX  and  VAX;  Silicon  Graphics  IRIS 
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Agenda 

Overview  Embedded  Real-Time  Reasoning? 
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Embedded  Real-Time  Reasoning  Systems 

Why? 
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Use  Iterative  Development  Process:  Concepts  -  Prototypes  -  Applications 
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•  Lack  Engineering  Rigor 

•  Require  Specialized  Computing  Platforms,  Languages  &  Engineers 


Embedded  Reasoning  vs.  Conventional  Systems 
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These  Networks  are  Coded  as  Conditionals  with  Numerous  Flags  and  Switches 
which  are  Difficult  to  Develop,  Understand  and  Maintain 


Knowledge  Based  Approach 
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Knowledge  Driven 

•  Fact  Assertions  Automatically  Focuses  &  Drives  Appropriate  Code  Execution 


Knowledge  Based  Software  Development 

Rapid  Prototyping 
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Knowledge  Based  Software  Development 

Requirements  »  Control  Logic  - ►  Emulation  - ►  Code  Generation - ►  Object  Code 


•  Supports  Tighter  Coupling  Between  the  Coders  and  Engineers 

•  Allows  Shorter  Development  Time 

•  Eliminates  AI  Code  From  the  Final  Product 
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Ada  Real-Time  Inference  Engine  (ARTIE)  Tool 

ARTIE  is  the  Main  Tool  that  Facilitates  Embedded  Reasoning  Software  Development 
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ARTIE  with  all  associated  tools  (parser,  interactive  user  interface,  debug/ 
monitoring  tools,  etc.)  25  KLOC 
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Technology  Development  Time  Lines 


October  21 , 1992  Boeing  Defense  &  Space  Group  - 13 - 


Demonstrated  Technology  Applications 
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Demonstrated  Technology  Applications 
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Automated  Sensor  Manager  Overview 
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Advanced  Man-Machine  Interfaces 


Automated  Sensor  Manager  Overview 
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Automated  Sensor  Manager  Overview 
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Real-Time  Embedded  Reasoning  Rick  Wilber  and  Rob  Ensey 
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Provides  Facilities  for  Storage  and  Retrieval  of  Multiple  Freeze  Frame  Sensor 
Images 


Real-Time  Embedded  Reasoning  Rick  Wilber  and  Rob  Ensey 
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Sensor  Manager  Reasoning  &  Targeting  Displays 


Real-Time  Embedded  Reasoning  Kick  Wilber  and  Rob  Ensey 
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Real-Time  Embedded  Reasoning  Status 


«<5 

b 

s*3 

•c 

o 

as 

•b 

B 

« 

I 


3 


B 

! 

OS 

I 

a 

Si 

E 


2 


C/3 

4— > 

c 

cj 

£ 

x 


60 

C 

‘E 

o 

C/3 

03 

4) 

Q£ 

*o 

4> 

C/3 

03 

CQ 

03 

-a 

< 


DO 

g 

*5 

o 

C/3 

03 

4> 

a: 

•o 

cj 

T3 

T3 

4J 

X) 

£ 


C/3 

CJ 

3 

_cr 

'£ 

X 

cj 

E- 

c 

CJ 

£ 

a. 

_c 

cj 

> 

CJ 

Q 

*o 

cj 

C/3 

03 

CQ 


C/3 

CJ 

CJ 

60 

’E. 

£ 

T3 

£ 

o 

CJ 

CJ 

H 

i 

03 

CJ 

o 

< 

T3 

CJ 

a: 

G 

CJ 

4-4 

X 

C 

03 

i— 

60 

CJ 

•*— < 

C/3 

C 

E 

i 

u 

o 

£ 

<4— . 

W 

CJ 

a 

• 

• 

60 

C 

*33 

C/3 

cj 

CJ 

o 

u. 
CL 
i—  «rj 
«  CJ 

a  cj 

£  e- 

e- 5/5 

CJ  JZ 

G  .BP 

c  ^ 

03 

C  “■ 

1 1 

60  2 
c 

’a. 
o 

4—t 

2 

CL 

“O  _ 

o 

«  = 

QC  < 


cj 

c 

CJ 

a 

CJ 

*a 

o 

U 


c 

3 

C/2 

<rv 

03 

O 

W 

o 


60 

C 

"33 

C/3 

CJ 

CJ 

O 


03 

C 

.2 

•4—* 

C 

o 

> 

c 

o 

U 

13 

C 

03 

< 

*n 

x 

>> 

ac 

CJ 

> 


CJ 

oj  15 
C.  v- 
<*-.  o3 
CQ  CL 


C/3 

e 

o 

a 

03 

CJ 


S  £ 
o 

£ 


C/3 

a 


•r  o 

«  £ 


PQ 

CQ 

H 

0* 

60 

G 

’£ 

o 

C/3 

03 

<J 

oc 

T3 

C 

03 

60 

G 

"35 

C/3 

CJ 

cj 

o 


T3 

4) 

3 

X) 


C/3 


O 

.  C/3 

c 

(U 

C/2 

«K 

2 

3 

O 

QC 

60 

G 

’£ 

c 

03 


c 

o 

*cw 

C/3 


T3 

4) 

"O 

T3 

CJ 

X 

£ 

CQ 

CJ 

£ 

P 

i 

C3 

4> 

a: 


03 

CJ 


60 

.£ 

'£ 

3 

03 


CJ 

4-4 

3 

O 

QC 


60 

E 

G 

CJ 

60 

03 

G 

03 


C 

o 

"E 

C/3 


03  O 

O  C 

PQ  4> 

C  £ 
>-V  CJ 
60 
03 
C 

4-  03 

2  S 

4-^ 

C  = 

o  .2 
£  £ 


T3 

CJ 


cj  x 

Q  5 


CL 

H 

QC 

C/2 

T3 

CJ 

C8 

£ 

o 

3 

< 


CL 

< 

c/2 

CD 

o 

"o 

o 

H 

s- 

4) 

C 

C 

03 


03 

CJ 

t- 

< 


03 

CJ 

c/2 

T3 

CJ 

CJ 

> 

”cj 

Q 


C/3 


60 

C 

’£ 

c 

=2  5» 


— 

3 

C 


-a 

G 

03 

C/3 

o 

O 

E- 


CJ 

X 

C/2 


CJ  •— 


CJ 

03 

P 

T3 

CJ 

■*-» 

CB 

i— 

60 

CJ 


60 

C 
•  «— » 

G 

O 

C/3 

03 

CJ 

Q£ 


£ 

cj  a 
X  ^ 
L- 


3 

UL 

u« 

,o 


CJ 

60 

03 

C 

03 


G 

O 

'vi 

C/3 


03 

CJ 


G 

_o 

c5 

i— 

60 

CJ 


w 

H 

Qi 

< 

ffl 

QQ 

H 

QC 


C/3 

C/3 

CJ 

CJ 

CJ 

3 

C/2 


October  21, 1992  Boeing  Defense  &  Space  Group  -2 5. 


Ongoing  Activities 

Continue  Refining  and  Applying  Embedded  Reasoning  Concepts 
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Exploit  the  Technology  in  Depth  on  Current  Projects 


APPENDIX  E 


Training  Control  &  Evaluation  (TC&E) 

Editor's  Notes 

BBN  has  successfully  used  an  iterative  prototyping  approach  to  perform  requirements 
analysis,  design,  and  Full-Scale  Engineering  Development  (FSED)  for  a  70  KSLOC 
Artificial  Intelligence  (AI)  application.  This  application.  Training  Control  &  Evaluation 
(TC&E),  is  a  valuable  example  of  the  success  that  can  be  achieved  by  rapid  prototyping  AI 
applications  with  Ada.  One  of  the  most  difficult  tasks  in  a  rapid  prototyping  requirements 
analysis  activity  is  the  transition  of  the  legacy  prototypes  and  knowledge  to  the  FSED.  The 
TC&E  experience  offers  some  valuable  insight  into  a  successful  transition  of  prototypes  from 
the  requirements  analysis  to  a  FSED  for  a  DOD-STD-2 1 67A  project. 

AI  Topics:  Blackboard  architecture,  goal-setting,  path-following,  progress  tracking,  obstacle 
avoidance,  target  selection 

Domain  Area:  Training,  simulation  and  modeling 

Language  Interfaces:  C  and  4GL 

Project  Status:  Fielded 

Size  of  Ada  Source  Code:  70  KSLOC 

Design/Development  Methodology:  Iterative,  rapid  prototyping 
Hardware  Platforms:  Sun 
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Training  Control  &  Evaluation  (TC&E) 


Dan  Massey 
BBN 

Cambridge,  Massachusetts 
617-873-3515 

BBN  recently  completed  development  of  a  major  distributed  simulation  system 
(called  AGPT)  for  the  German  Army.  This  system  was  built  in  Ada,  and  includes  a  group  of 
CSCIs  (abo  '  70  KSLOC  of  Ada)  which  support  the  instructor's  interface  to  the  system  and 
provide  interactive  control  over  simulated  elements  of  a  virtual  environment  which  are 
endowed  with  the  ability  to  adapt  and  react  to  a  changing  tactical  situation  in  an  "intelligent” 
manner.  This  part  .;f  AGPT,  called  the  Training  Control  &  Evaluation  (TC&E)  segment,  is 
based  on  a  segment  of  the  SIMNET  system  previously  implemented  in  the  US  for  DARPA, 

The  original  SIMNET  implementation  (called  Semi-Automated  Forces,  or  SAF)  was 
built  as  a  prototype  in  LISP  on  Symbolics  workstations  and  gradually  ported  to  C  over  a 
number  of  years.  The  present  implementation  of  SAF  is  about  300  KSLOC.  These  program 
sizes  are  not  comparable  since  TC&E  includes  functionality  not  present  in  SAF,  and  vice- 
versa.  In  addition,  about  65  KSLOC  of  C  libraries  (counted  in  the  SAF  total)  are  used  in 
TC&E  to  support  certain  system  housekeeping  functions  (network  connections,  matrix  and 
quaternion  math,  real-time  terrain  analysis)  that  were  common  to  both  systems  and  for  which 
assured  consistency  of  implementation  was  deemed  to  be  very  important. 

Although  the  "higher-level  cognitive  functions"  simulated  in  TC&E  are  entirely 
written  in  Ada,  including  such  functions  as  goal-setting,  path-following,  progress  tracking, 
obstacle  avoidance,  target  selection,  etc.,  and  involve  the  systematic  application  of  a  set  of 
rules  (some  impletn.  ■'ted  explicitly  as  Ada  procedures  and  others  provided  in  a  more  general 
rule  definition  language,  the  ability  of  the  system  to  operate  in  real  time  does  depend  on  the 
fact  that  the  most  computationally  intensive  parts  of  terrain  analysis  (computing  the  lines  of 
visibility  surrounding  a  point  in  the  virtual  world)  are  implemented  in  carefully  crafted  and 
tightly  coded  C. 

Most  of  TC&E  Ada  (about  80%)  was  built  bottom-up,  prior  to  agreement  on  either  a 
system  specification  or  a  definite  detailed  design.  Essentially,  the  core  of  TC&E  is  a  rapid 
prototype,  written  in  Ada,  which  has  been  subsequently  refined  through  a  series  of 
incrementally  enhanced  releases.  During  the  early  phases  of  development  (prior  to 
agreement  on  the  system  specification)  a  number  of  pre-release  versions  of  parts  of  TC&E, 
up  to  about  50%  functionality,  were  demonstrated  and  evaluated  in  depth  with  the  client  to 
help  in  finalizing  the  specification. 

The  TC&E  prototype  (about  80%  functionality)  became  an  operational  definition  of 
segment  design  that  was  used  in  the  full-scale  engineering  development  of  the  production 
code.  Through  an  agreed  process  of  CSU  documentation  and  test,  large  parts  of  the 
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prototype  code  were  formally  "adopted"  into  the  final  product.  Other  parts  were  significantly 
reworked  to  improve  testability,  reusability,  and  reliability  before  adoption.  Only  about  20% 
of  the  TC&E  segment  code  was  produced  de  novo  during  FSED.  These  pans,  largely 
enhancements  to  the  prototype  backbone,  were  specified,  designed,  and  documented  through 
a  relatively  formal  engineering  change  process. 

We  consider  that  the  entire  prototype  development  phase  of  TC&E  (which 
represented  more  than  two  thirds  the  calendar  time  and  labor  to  obtain  the  final  product)  was 
a  requirements  analysis  activity  (in  the  sense  of  Royce’s  water-fall  life  cycle)  and  covered  the 
inner  cycles  of  a  spiral  development  process  (in  the  sense  of  Boehm). 

TC&E  Acronym  List 

CSCI  Computer  Software  Configuration  Item 

CSU  Computer  Software  Unit 

DARPA  Defense  Advanced  Research  Agency 

FSED  Full  Scale  Engineering  Development 

KSLOC  Thousands  of  Source  Lines  of  Code 

SAF  Semi- Automated  Forces 

SIMNET  Simulation  Network 

Training  Control  &  Evaluation 


TC&E 
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Ada  9X  Issues  for  A I  Systems 
Editor's  Notes 

Henry  Baker  chaired  this  panel  at  the  workshop  and  led  some  very  thought  provoking 
discussions  about  Ada  9X  issues  for  AI.  Henry  describes  the  classic  implementation 
characteristics  of  traditional  AI  programs  -*  late  binding,  large  address  spaces,  extensive  use 
of  shared  memory  for  communication,  extremely  large  programs,  very  dynamic  problem 
spaces,  complex  and  non-homogeneous  data  structures  -*  and  offers  suggestions  for  an  AI 
Annex  that  includes  procedures  as  parameters,  a  pragma  Garbage_Collection,  provides  more 
general  type  initialization,  generic  types,  and  type  instantiation  parameters.  His  issues  paper 
and  briefing  are  titled  Ada9X  Issues  for  AI  Systems. 
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Ada9X  Issues  for  Al  Systems 

Henry  G.  Baker 

Nimble  Computer  Corporation,  16231  Meadow  Ridge  Way,  Encino,  CA  91436 
(818)  501-4956  (818)  986-1360  (FAX) 

We  must  offer  suggestions  to  the  Ada-9X  Committee  for  allowing/enhancing  AI  programs  in  Ada. 


Defense  Al  Applications  in  Ada-83?  In  Ada-9X?  What  are  the  issues? 

Can  AI  programs  be  successfully  written  and  deployed  in  Ada-83?  Do  the  changes  contemplated  in 
Ada-9X  make  the  writing  and  deployment  of  AI  programs  any  easier?  Do  there  still  exist  major 
"gotchas"  in  Ada-9X  which  will  seriously  decrease  the  performance  and/or  increase  the  cost  of 
developing  and  deploying  AI  programs  in  Ada? 

Al  programs  are  some  of  the  largest  programs  around,  in  terms  of  lines  of  code,  complexity,  cost, 
etc.  I.e.,  they  are  large  programs,  but  small  amounts  of  data,  relative  to  more  traditional  embedded 
systems  programs.  E.g.,  compiled  program  text  of  large  AI  system  can  take  20  megabytes, 
whereas  "large"  non-AI  programs  use  1-2  megabytes  for  both  program  and  data  (not  counting  large 
passive  databases  such  as  terrain  mapping  databases). 

What  are  the  design,  implementation  and  maintenance  implications  of  such  large  programs  in  an 
Ada  environment?  E.g.,  small  changes  may  cause  massive  recompilations — it  could  take  hours  or 
days  to  "make"  a  new  version  of  a  system.  With  a  program  of  this  size,  is  it  ever  "delivered"?  If 
only  10'3  of  such  a  program  is  changed  annually,  this  may  still  be  10,000  lines  of  code  changed  per 
year.  Rumors  exist  of  massive  software  changes/fixes  for  the  Patriot  missile  system  while  the  unit 
was  already  in  battle.  Significant  changes  may  be  required  in  a  "Pilot's  Associate"  program  for 
every  mission.  In  other  words,  "program"  may  have  become  "data",  to  be  loaded  for  each  and 
every  mission.  Does  this  mean  that  "dynamic  loading"  a  la  Berkeley  Unix  or  Unix  V  Rel.  4  should 
become  part  of  an  Ada  run-time  system?  Does  this  require  a  "persistent  Ada  heap",  a  la  current 
"object-oriented  databases"? 

A  "signature"  characteristic  of  AI  programs  is  their  "late  binding"  of  control  constructs,  which  are 
universally  implemented  by  means  of  first-class  function  closures.  These  closures  are  dynamically 
constructed  functions  which  can  be  passed  as  arguments,  returned  as  values,  and  stored  into  data 
structures  as  values.  Ada-83  was  expressly  forbidden  by  its  Steelman  Requirements  to  have  no 
such  capability.  Ada-83  offers  generic  functions  and  procedures,  which  can  emulate  some,  but  not 
all,  of  these  late  binding  constructs.  Do  the  capabilities  of  Ada-9X  provide  enough  relief  to  satisfy 
the  AI  developer,  or  should  we  send  the  Ada-9X  team  back  to  the  drawing  board? 

AI  people  have  been  requesting  garbage  collection  for  Ada  at  least  since  1980  [Schwartz80],  yet  no 
vendor  provides  it,  and  Ada  compiler/runtime  validation  does  not  require  garbage  collection.  Yet 
GC  is  an  extremely  valuable  tool  in  allowing  the  decomposition  of  large  systems  without  increasing 
the  probability  of  failure  due  to  dangling  references.  Such  dangling  references  are  becoming  more 
and  more  likely  with  the  dramatic  increase  in  pointer-based  programs  due  to  the  popularity  of 
"object-oriented"  programming.  Can  garbage  collection  be  emulated  on  top  of  Ada  with  enough 
efficiency  to  support  the  heavy  computational  demands  of  AI  programs? 

Traditional  AI  systems  require  a  large  address  space  and  the  shared-memory  paradigm.  Yet  many 
embedded  systems  are  designed  with  hardware  that  supports  a  distributed-memory/message¬ 
passing  model,  and  it  may  be  quite  difficult  to  map  AI  programs  onto  these  platforms.  The  Ada 
parallel  process  model  clearly  prefers  an  explicit  exchange  of  information  via  the  rendezvous 
mechanism,  and  only  grudgingly  supports  the  notion  of  asynchronous  access  to  shared  data.  Yet 
the  most  popular  model  for  parallel,  embedded  real-time  AI  systems  is  the  blackboard  model, 
which  has  at  its  core  a  database  shared  and  asynchronously  updated  by  all  processes! 
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Although  Ada  was  standardized  in  1983,  production  quality  compilers  were  not  available  until  the 
1986-87  time  frame,  and  significant  bugs  are  still  prevalent  in  Ada83  compilers  today.  For 
example,  generics  could  not  be  used  reliably  in  the  first  generation  of  Ada83  compilers,  and 
"storage  leaks"  continue  even  in  today’s  Ada83  runtime  systems.  Thus,  it  seems  prudent  to 
recognize  that  it  may  be  1995  before  debugged,  reliable  compilers  and  runtime  systems  are  available 
for  Ada-9X.  In  this  case,  another  generation  of  weapons  systems  will  be  developed  in  Ada83 — 
i.e.,  they  will  not  be  able  to  take  advantage  of  any  of  the  newer  Ada-9X  capabilities.  Since  Al 
capabilities  are  being  put  into  systems  today,  what  is  the  near-term  effect  of  doing  this  in  Ada83  v. 
Ada-9X?  What  are  the  long-term  effects — i.e.,  efficiency,  maintenance,  obsolete  protocols,  etc. — 
of  this  delay? 

Some  "100%  Ada"  projects  are  using  Ada  as  "just  another  language",  to  be  loaded  into  a  separate 
address  space  on  a  classical  operating  system  with  multiple  address  spaces.  Any  synchronization 
between  the  separate  address  spaces  is  implemented  by  means  of  non-Ada  capabilities — e.g., 
locking  in  a  "file"  system.  The  Ada  strong  typing  system  is  side-stepped  by  reading  and  writing  to 
external  "files”.  Worst  of  all,  Ada  run-time  checks  within  an  "application"  (i.e.,  address  space)  are 
disabled,  with  any  protection  provided  by  the  hardware — e.g.,  "bus  error".  Do  such  system 
designs  conform  to  the  spirit,  as  well  as  the  letter,  of  the  Ada  law,  or  are  they  a  pragmatic  solution 
to  an  inflexible  language  standard?  Perhaps  such  systems  recognize  the  inevitable  need  to  interface 
Ada  with  COTS  technologies — most  likely  C,  C++  or  even  Lisp(!). 

Issues  of  ancillary  standards  and  tools.  Are  the  MIL-STD's  for  software  design,  documentation, 
implementation  and  testing  appropriate  for  AI  programs,  or  are  they  too  rigid?  Can  the  complex 
notions  of  Al  programs  even  be  expressed  in  these  "design  methodologies"  and  "design  tools",  or 
are  new  methodologies  and  tools  required.  Do  the  proposed  documentation  and  coding  standards 
put  the  AI  programmer  into  a  straight-jacket  (assuming  that  Ada  itself  hasn't  already)?  Are  the 
APSE/KAPSE/...  tools  part  of  the  solution,  or  part  of  the  problem? 

Are  real-time  AI  programs  a  mirage?  Can  an  AI  program  ever  be  expected  to  always  respond  within 
a  fixed  latency,  or  must  we  start  planning  for  only  stochastic  response  latencies?  What  sorts  of 
scheduling  capabilities  do  AI  programs  require  beyond  those  useful  for  other  real-time  programs? 

Overall  Goal  of  Workshop  and  Summary 

Potential  contractor/developers  of  defense  software  systems  have  little  incentive  to  make 
investments  in  standards  or  tools  for  the  uncertain  likelihood  of  future  contracts.  Since  AI 
capabilities  are  new,  there  is  no  established  pool  of  experience  in  the  defense  software  contracting 
industry  which  can  fight  for  the  language  changes  and  tools  which  will  make  AI  programming 
easier,  cheaper  and  more  effective.  There  are,  on  the  other  hand,  major  established  groups  to  argue 
for  better  signal  processing  support,  better  decimal/mainframe  support,  better  network  support, 
better  real-time  support  for  traditional  software  control  loops,  etc.  It  is  therefore  likely  that 
significant  "holes"  exist  in  the  Ada  language  and  infrastructure,  which  will  only  become  evident 
later,  when  projects  become  late  and  costs  balloon. 

The  overall  goal  of  this  workshop  is  a  document  which  clearly  states  the  requirements  for 
programming  languages  to  support  real-time  embedded  AI  programs  for  defense  applications. 
These  requirements  need  to  be  prioritized,  and  the  consequences  and  costs  of  not  meeting  the 
requirements  need  to  be  estimated.  Since  modem  warfare  puts  the  ultimate  premium  on  up-to-date 
intelligence,  efficient  resource  allocation,  and  pin-point  accuracy,  AI  will  play  a  pivotal  role  in 
making  sure  that  the  weapons  are  located  at  the  right  place  and  the  right  time,  and  used  against  the 
right  target  with  the  appropriate  ammunition  with  sufficient  accuracy  and  concentration  to  knock  out 
the  target  once,  but  only  once.  We  have  to  make  sure  that  we  are  fighting  the  next  war  rather  than 
the  previous  war. 
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Why  are  AI  people  here  discussing  Ada9X  now? 
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Ada9X  Issues  for  AI  Systems 

•  Can  AI  programs  be  successfully  written  and  deployed  in  Ada? 

•  Does  Ada9X  make  writing/deploying  AI  programs  easier? 
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•  "Generational"  GCs  which  are  good  for  development  envs. 
are  not  appropriate  for  embedded  systems 
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APPENDIX  G 


MESSAGE  FROM  THE 

APPLICATIONS  EXPERIENCES  AND  LESSONS  LEARNED 

PANEL  CHAIR 


SIGAda  Artificial  Intelligence  Working  Group 
Summer  '92  SIGAda  Workshop 


Janet  Faye  Johns 


The  MITRE  Corporation 
Bedford,  Massachusetts 
617-271-8206 

This  paper  assesses  the  current  state-of-the-art  for  Artificial  Intelligence  (AI) 
applications  development  processes.  The  human  and  safety  reasons  why  we  need  to 
"engineer"  AI  applications  is  discussed  and  information  about  the  current  processes  used  to 
develop  knowledge-based  systems  is  presented.  Issues  associated  with  requirements 
analysis,  design  methodologies,  development  techniques,  test  and  validation,  and 
maintainability  are  discussed  for  AI  applications  and  for  AI  with  Ada  applications. 

Viewgraphs  are  in  this  paper  with  an  accompanying  discussion  section  that  is  divided 
into  two  parts.  The  first  states  current  issues  while  the  secord  pan  provides  a  summary  of  the 
related  workshop  discussions. 
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APPLICATIONS  EXPERIENCES  AND  LESSONS  LEARNED  PANEL 


MESSAGE  FROM  THE  PANEL  CHAIR 


BACKGROUND 

During  the  past  year,  I  have  been  collecting  information  about  existing  Artificial 
Intelligence  (AI)  applications  developed  with  the  Ada  programming  language  for  the  SIGAda 
Artificial  Intelligence  Working  Group's  (AIWG)  first  annual  survey  |3J.  This  was  a 
challenging  task  for  a  number  of  reasons,  but  I  have  come  to  believe  that  the  major 
impediment  to  my  data  collection  efforts  is  the  state-of-the-art  of  developing  Al  applications. 
The  basic  nature  of  specifying  and  developing  Al  systems  is  a  major  stumbling  block  to 
formulating  detailed  software  metrics  and  other  measures  that  are  so  common  for 
"engineered"  systems.  AI  systems  generally  begin  with  few-  documented  requirements  and 
are  developed  with  a  series  of  evolutionary  prototypes.  Typical  software  metrics  are  not 
readily  available  for  Al  systems.  This  fact  led  me  to  investigate  the  current  processes  used  to 
develop  AI  applications  and  the  problems  of  applying  current  software  engineering  principles 
to  AI  applications. 

This  briefing  was  presented  at  the  general  session  of  the  Summer  '92  SIGAda 
conference.  I  presented  this  material  as  a  devil’s  advocate  challenge  that  stated  the  issues  in 
an  effort  to  generate  open  discussions  about  the  issues.  These  and  many  other  issues  were 
discussed  during  the  workshop.  For  the  publication  of  this  briefing  in  the  workshop 
proceedings,  I  have  added  relevant  information  from  the  workshop  discussions  to  the 
briefing.  My  briefing  combined  with  my  interpretation  of  the  workshop  discussions  is  the 
method  I  have  chosen  to  document  the  workshop  discussions  for  these  proceedings. 
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software  engineering  principles  when  we 
write  software  to  solve  our  problems. 


The  1991  AIWG  industry  survey  |3|  provided  empirical  evidence  that  the  Ada  programming  language  has  been  used 
successfully  to  develop  Artificial  Intelligence  (AI)  applications.  The  difficulty  encountered  in  collecting  software  metrics 
for  the  1991  AIWG  survey  raised  many  questions  about  the  current  state-of-the-art  for  AI  requirements  analysis,  desien. 
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Nancy  Leveson,  May  l992,High  Pressure  Steam  Enqines_and 
Computer  Software  Keynote  address  at  the  14th  International 
Conference  on  Software  Engineering,  p  2  -  14  of  the  Proceedings. 
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We  must  understand  the  human  impact  of  our  inventions. 
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Theorem:  An  optimal  software  engineering  process  exists  for  all  Artificial  Intelligence  problems 

Our  greatest  challenge  is  the  application  of  software  engineering  principles  to  the  development  of  AI  applications.  This 
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7)  Scalability  from  the  requirements  analysis  and  design  prototypes  to  a  full  scale  system  is  an  ill-defined  area  for  Al 
software. 
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This  “design  for  evolution"  is  accomplished  by  conforming  to 
standards  (UNIX,  FDDI,  Ada,  X  Windows)  and  designing  the 
interconnections  with  sufficient  bandwidth  to  allow  for 
faster  components  as  they  become  qualified  for  space. 


Our  Challenge:  Software  Engineering  with  Ada  and  A! 
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Phillippe  Collard  and  Andre  Goforth,  November/December  1988, 
Knowledge  Based  Systems  and  Ada:  An  Overview  of  the  Issues. 
Ada  Letters,  p  72  -  81. 
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Phillippe  Collard  and  Andre  Goforth,  November/December  1988, 
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advantage  is  not  realized  in  practice. ..." 

Xiaofeng  Li,  September  1991,  What’s  so  bad  about  rule-based  programming?, 
IEEE  Software,  Vol  8,  No  5,  p  133  -  105.  Paul  Sanders  responded  to  this  article 
in  the  January  1992  issue  of  IEEE  Software. 


The  difficulties  inherent  in  A1  requirements  analysis  inevitably  lead  to  problems  with  the  testing,  verification,  and 
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David  Hamilton,  Keith  Kelly,  and  Chris  Culbert,  1991, 

State-of-the-Practice  in  Know)edqe_Based  System  Verification  and  Validati 
Expert  Systems  with  Applications,  Vol  3,  No  4,  p  403  -  410. 


Case  Study:  Accuracy  of  Existing  Systems 

Gilbert,  Hamilton,  and  Kelly  documented  the  experiences  and  problems  encountered  by  knowledge  based  systems 
developers  and  users.  In  their  paper,  the  terms  knowledge  based  systems  and  expert  systems  were  used  interchane 
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Case  Study:  Design  Methodologies 
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Case  Study:  Operational  Prototypes 

As  discussed  in  the  survey,  70%  of  the  systems  were  operational  and  the  rest  were  considered  prototypes  but  some  of 
these  prototypes  had  operational  users.  The  respondents  could  not  clearly  distinguish  between  prototypes  and  operational 
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